This invention relates to the steering of monaural sources of sound to any desired location in space surrounding a listener by using the head-related transfer function (HRTF) and compensating for the crosstalk associated with reproduction on a pair of loudspeakers.
More particularly, the invention provides an efficient system whereby any number of monaural sound sources can be steered in real time to any desired spatial locations. The system incorporates compensation of the loudspeaker feed signals to cancel crosstalk, and a new technique for interpolation between measured HRTFs for known sound source locations in order to generate appropriate HRTFs for sound sources in intermediate locations.
The following are references to related patents and papers in the art:
1. Atal B. S. and Schroeder, M. R., xe2x80x9cApparent Sound Source Translator,xe2x80x9d U.S. Pat. No. 3,236,949, Feb. 22, 1966.
2. Blauert, J., xe2x80x9cLateralization in the Median Plane,xe2x80x9d Acustica vol. 22 pp. 957-962, 1969.
3. Blauert, Jens, xe2x80x9cSpatial Hearing,xe2x80x9d J. S. Allen, transl., MIT Press, Cambridge, Mass., 1983, 1996.
4. Cooper, D. H., and Bauck, J. L., xe2x80x9cHead Diffraction Compensated Stereo System,xe2x80x9d U.S. Pat. No. 4,893,342, Jan. 9, 1990.
5. Cooper, D. H., and Bauck, J. L., xe2x80x9cHead Diffraction Compensated Stereo System with Optimal Equalization,xe2x80x9d U.S. Pat. No. 4,910,779, Mar. 20, 1990.
6. Cooper, D. H., and Bauck, J. L., xe2x80x9cHead Diffraction Compensated Stereo System with Optimal Equalization,xe2x80x9d U.S. Pat. No. 4,975,954, Dec. 4, 1990
7. Cooper, D. H., and Bauck, J. L., xe2x80x9cHead Diffraction Compensated Stereo System,xe2x80x9d U.S. Pat. No. 5,034,983, Jul. 23, 1991.
8. Cooper, D. H., and Bauck, J. L., xe2x80x9cHead Diffraction Compensated Stereo System,xe2x80x9d U.S. Pat. No., 5,136,651, Aug. 4, 1992.
9. Cooper, D. H., and Bauck, J. L., xe2x80x9cHead Diffraction Compensated Stereo System with Loud Speaker Array,xe2x80x9d U.S. Pat. No. 5,333,200, Jul. 26, 1994.
10. Cooper, D. H., and Bauck, J. L., xe2x80x9cProspects for Transaural Recording,xe2x80x9d J. Audio Eng. Soc., Vol. 37, pp. 3-19, January/February 1989.
11. N. Fuchigami et al., xe2x80x9cMethod for Controlling Localization of Sound Images,xe2x80x9d U.S. Pat. No. 5,404,406, 1994.
12. Shaw, E. A. G, and Teranishi, R., xe2x80x9cSound Pressure Generated in an External Ear Replica and Real Human Ears by Nearby Point Sources,xe2x80x9d J. Acoust. Soc. Am., vol. 44, pp. 240-9, 1968.
13. Wright, D., Hebrank, J. H., and Wilson, B., xe2x80x9cPinna Reflections as Cues for Localization,xe2x80x9d J. Acoust. Soc. Am., Vol. 56, pp. 957-962, 1974.
14. Blumlein, A. D., xe2x80x9cImprovements in and Relating to Sound Transmission,xe2x80x9d British Patent No. 394,325, filed Dec. 14, 1931, issued Jun. 14, 1933.
15. Butler, R. A., and Belendiuk, K., xe2x80x9cSpectral Cues Utilized in the Localization of Sound in the Median Sagittal Plane,xe2x80x9d J. Acoust. Soc. Am., Vol. 61, no. 5, pp. 1264-1269, 1977.
16. Widrow, B., and Strearns, S., xe2x80x9cAdaptive Signal Processing,xe2x80x9d Prentice-Hall, 1985.
17. Eriksson, L., xe2x80x9cDevelopment of the Filtered-U Algorithm for Active Noise Control,xe2x80x9d J. Acoust. Soc. Am., Vol. 89, pp. 257-265, 1990.
18. Eriksson, L., xe2x80x9cActive Attenuation System with On Line Modeling of Speaker, Error Path and Feedbackxe2x80x9d U.S. Pat. No. 4,677,767, Jun. 30, 1987.
Stereophonic sound reproduction systems employ psychoacoustic effects to provide a listener with the impression of a multiplicity of separate real sound sources, for example musical instruments and voices, positioned at several distinct locations across the space between the left and right loudspeakers which are usually placed symmetrically to either side in front of the listener.
Pairwise mixing is an example of an early technique for producing such an impression. The sound is provided to both channels in phase, with an amplitude ratio following a sine-cosine curve as a sound source is panned from one side of the listener to the other. While this approach has been a generally accepted one, it has proved deficient in several ways; the apparent location of the sound is not stable when the listener""s head moves, and sounds between the loudspeakers appear to be above the line joining them More recent research in psychoacoustics has shown that when sound is diffracted round the listener""s head, in general the left and right ears hear different transfer functions applied to the sound; an impulse will reach the far ear later than the near ear, and the shadowing provided by the head will alter the amplitude of the sound reaching the far ear relative to that reaching the near ear, the amplitude differences being a complicated function of frequency. These functions are termed xe2x80x9chead-related transfer functionsxe2x80x9d and include effects due to reflections of sound by the pinnae and torso of the individual listener.
A somewhat simplified model of the head as a sphere, with orifices at left and right representing the ears and without the equivalent of pinnae, can be used to derive a generic HRTF theoretically or through numerical analysis. Because there are no pinnae, there is no difference between the HRTFs for sounds to the front of or equally to the rear of the lateral center line. Also, the lack of pinnae and torso modifications precludes differences due to the height of the sound source above the plane containing the ears. Nevertheless, the xe2x80x9cspherical headxe2x80x9d model has at least pointed the way to understanding the subtleties of HRTF effects.
An alternative reproduction method to stereophony is binaural recording, which typically employs a xe2x80x9cdummy headxe2x80x9d or manikin of a generic character, with pinnae and torso effects included, which has HRTFs that may be considered xe2x80x9caverage.xe2x80x9d Microphones are placed in the ear canals of the dummy head to record the sound, which is then reproduced in the listener""s ears using headphones. Because individuals differ in head size, placement and size of the ears, etc., each listener would obtain the most realistic binaural reproduction if the dummy head used for recording were an exact replica of his own head. The differences are sufficient that some listeners may have difficulty in differentiating the front or rear locations of some sounds reproduced this way. A further disadvantage of this method is that when reproduced over loudspeakers, sounds intended for reproduction only in the left or right ear are heard differentially by both ears, and the HRTFs corresponding to the loudspeaker locations are superimposed onto the sounds, contributing to unnatural frequency response effects.
Various methods for cancellation of the crosstalk between the loudspeakers have been devised, and this art is assumed in this patent application. Thus, the reproduction of binaurally recorded sound could take place either on headphones or through loudspeakers with the crosstalk cancellation method applied in the latter case.
In order to produce realistic recording and reproduction of sounds in specific locations relative to the listener, it is desirable to have a method which can simulate any location of a monaural source within the sound stage reproduced through a pair of loudspeakers. Since pairwise mixing has been found to have considerable drawbacks, a method that employs the known psychoacoustical effects of HRTFs is significantly better. Furthermore, such methods can also simulate sound locations to the sides and rear of the listener.
Although digital filtering can be used to provide these complex enhancements of the sound signals prior to mixing down onto two-channel media, for reproduction on a pair of loudspeakers, the cost and complexity of such filtering is often an obstacle to obtaining the most realistic reproduction. Therefore, the efficiency of the method must also be considered, as a method using fewer coefficients to obtain the same result will typically be lower in cost.
The present invention, therefore, provides an efficient system and method whereby any number of monaural sound sources can be steered to any desired location in space, either in real time or in another specified manner such as mixing down from multi-track recordings. The listener will be given the impression that there exist xe2x80x98realxe2x80x99 sources of sounds at these locations. The method is based on the head related transfer function (HRTF) and compensates for the crosstalk associated with the speakers.
In one embodiment, electronic signal steering apparatus converts a monaural signal derived from a sound source into left and right signals which drive corresponding headphones on a listener""s head, so that the listener experiences the impression that the sound source is at a specific location relative to his head, this effect being achieved by filtering the monaural signal using transfer functions equivalent to the HRTFs that would result from placing the actual sound source at the specified location relative to the listener.
Other embodiments to be described include compensation for loudspeaker crosstalk in the filters, so that the sound may be reproduced on loudspeakers and the listener may still perceive the sound as coming from the specified location.
An advantage of the invention is that it employs measured HRTFs obtained with a standard dummy head and incorporates a technique for interpolation between measured HRTFs to obtain an HRTF corresponding to a location where there is no measured HRTF available.
A further advantage of the invention is the use of Sigma and Delta filters to give positional cues for monaural sound sources.
Another advantage of the invention is the buffer schema used to minimize the transient effects of switching between positional filters when a sound source is in apparent motion.
Another advantage claimed for the invention is that only two filters are required whether loudspeakers or headphones are used, by incorporating into these filters the crosstalk cancellation required for loudspeaker reproduction in addition to the HRTF Sigma and Delta filtering to be described.
Another advantage of the invention is that by preserving the spectral peaks and notches produced by the pinnae and torso of the dummy head, more natural reproduction is obtained than for methods employing equalization according to Cooper and Bauck.
The invention provides a further advantage in its ability to calculate the approximated concatenated HRTF filters in real time using an adaptive filtering process.
The invention may also be advantageous in providing a method and system for generating more realistic spatial sound effects from music originated in a synthesizer or computer which otherwise no satisfactory spatial rendering exists.